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Abstract 

Based on a trend study of 970 graduating seniors, this 
paper presents a model for designing senior survey 
research studies to achieve optimum impact on 
assessment and policy development. The paper 
demonstrates how the link between research and policy 
was achieved through the conceptual organization, design 
and statistical analyses of the study. Confirming findings 
from previous research, this study identified some unique 
predictors of the two major dependent variables. The 
perception of having gained in-depth knowledge of afield 
was the strongest predictor of overall satisfaction, while 
satisfaction with the campus social life was the strongest 
predictor of the likelihood of choosing the same college 
again. 

Introduction 

Purpose . This paper presents a model for designing 
senior survey research studies to achieve optimum impact 
on assessment and policy development. Based on a 
completed senior survey trend study, the paper explains 
how the link between research and policy was achieved 
by conceptually organizing the study in relation to the 
institution’s mission, the goals of the undergraduate 
curriculum and the structure of undergraduate programs; 
by addressing policy relevant questions in the analyses; 
and by developing strategic policy recommendations to 
capitalize on the college’s strengths and enhance areas 
in need of improvement. 

Major research questions addressed in the study 
include the following: 

• To what extent do graduating seniors think college 
enhanced their abilities? 

• How satisfied are graduating seniors with college 
academic experiences and student life? 
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• How does assessment of college impact and 
satisfaction vary by gender and citizenship? 

• What are the significant predictors of graduating 
seniors’ overall satisfaction? 

• What are the significant predictors of seniors’ re- 
evaluation of their college choice? 

Review of the Literature 

Literature on the assessment of undergraduate 
education provides the foundation and context for the 
present study. Relevant themes from this literature 
include: designing assessment to measure college impact; 
incorporating students’ voice in assessment; using self- 
reported data to assess gain; understanding differences 
in assessment by student characteristics; and identifying 
predictors of overall satisfaction and re-evaluation of 
college choice. 

Designing assessment to measure college impact . 

One of the critical principles of good practice in 
assessment is that programs have clear, explicitly stated 
purposes (AAHE, 1992). As Banta, Lund, Black, and 
Oblander (1996) observe, an institutional mission 
statement is not sufficient as a basis for a comprehensive 
assessment program. The mission and values must be 
translated into specific and realistic goals for each 
academic program and student service to represent the 
direction in which faculty and administrators wish to see 
students grow and develop. In agreement with this view, 
Ratcliff (1995) recommends replacing outlines of course 
content with specific cognitive goals as a way to influence 
how faculty organize learning and promote student growth. 

Evaluating college in terms of student growth is a 
basic premise of college impact theories. Such theories 
assume that a primary purpose of a college education is 
to enhance student growth, and that higher education 
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institutions should be measured in terms of their impact 
on students. Tam (2002) elaborates on the implications 
of this theoretical perspective. 

This college impact approach to gauge the 
effects of the college on student outcomes is 
consistent with the view that higher education 
can literally transform one’s self-image, equip 
the i ndividual w ith m ore s kills, b uild o n t he 
basis of the knowledge that the individual had 
before arrival and change attitudes and 
assumptions, (p. 217) 

Based on empirical evidence of the impact of college 
on students, Pascarella and Terenzini (1995) claim that 
college facilitates a broad range of desirable changes 
that do not occur to the same extent to similar individuals 
who don’t attend college. Terenzini, Springer, Pascarella 
and Nora (1995) report that academic and non-academic 
experiences both separately and jointly affect student 
learning, thus providing evidence that college’s effects 
are holistic. 

Incorporating students’ voice in assessment Given 
the potential influence of college on students’ 
development, it seems logical to consider students’ 
perception of college impact and to include their voice in 
assessment studies. Ewell (1983) proposed that by 
using student satisfaction assessment results to shape 
institutional reform, institutions are validating the 
importance of the students’ “voice.” Supporting this view, 
Lingrell (1992) observed that because the ultimate 
beneficiary of assessment is the student it only seems 
appropriate to ask the student what he or she thinks. 

However, Twombly (1992) noted that the meaning 
students attach to the curriculum is a sadly neglected 
area of study. More recently, Keller-Wolff, Eason, and 
Hinds (2000) also reported that despite the comprehensive 
studies of curriculum and its affect on students, the 
student perspective is largely absent from the literature. 
The consequences of this omission are significant. 

In her discussion on “Moving Assessment Forward: 
Enabling Conditions and Stumbling Blocks,” Banta (1997) 
observed that insufficient involvement of students in 
assessment can contribute to its downfall. Further, Barr 
and Tagg (1995) claim that the lack of student voice in 
assessing the impact of curriculum is inconsistent with 
higher education’s shift from a paradigm that emphasizes 
teaching to one that emphasizes learning. 

Proposing a way to change the situation, Lingrell (1992) 
identified the senior survey as an ideal vehicle for 
incorporating students’ voice. He suggested that senior 
survey assessment information could be used to: 
recognize outstanding programs and teaching; improve 
admissions, instruction, and retention; and prepare for 



accreditation review. In this way, annual senior surveys 
may serve as a catalyst toward institutional reform. Nelson 
and Johnson (1997) provide examples illustrating how 
senior surveys may serve as vehicles for achieving 
program improvement in various areas, including the 
curriculum, career services and student advising. Cheng’s 
(2001) research provides an excellent example of how 
an externally developed senior survey may be used to 
construct a model for assessing student collegiate 
experience and producing outcome measures in an 
institution’s assessment effort. 

Gardner and Van der Veer (1998) identify the senior 
year as a unique time to elicit student feedback. Drawing 
on anthropological terminology, the authors describe 
seniors as being in a ‘liminal’ state in which they can 
reflect on the experiences they are about to leave and 
anticipate the world of work or graduate school they are 
about to enter. By virtue of this transitional state, seniors 
can provide more complex and insightful perspectives 
than might be gained from students during or after their 
college years. Further, the authors contend that surveys 
may generate valuable information about program 
effectiveness. 

Two major assessment topics typically addressed in a 
senior survey include student perceptions of what they 
have learned and gained through their college experience 
and student satisfaction with programs and services. 
With regard to perceived learning and benefit gained 
from college, Pascarella and Terenzini (1991) claim that 
the breadth of change is the most striking phenomenon 
in how college affects students, not only in factual 
knowledge but also in terms of general cognitive and 
intellectual skills and value, attitudinal, psychosocial, and 
moral dimensions. More recently, Pascarella and Terenzini 
(1998) also argue that the demographic, institutional, and 
economic changes that have occurred during the last 
three decades require a fundamental change both in how 
we think about what it means to go to college and in what 
methodologies we use to assess college impact. 

In addition to addressing perceived growth, senior 
surveys also typically elicit student satisfaction with their 
experience. This assessment topic is important both to 
the health of the institution and to student success. 
Shreiner and Juillerat (1997) recommend that higher 
education institutions give serious consideration to 
students’ satisfaction results as a way to recognize the 
institutional strengths as well as areas in need of 
improvement. Such assessments can assist institutions 
in establishing annual benchmarking of their student 
population; tracking the impact of new initiatives on student 
satisfaction; and identifying current strengths for 
recruitment activities. Further, these assessments can 
provide regular feedback to an institution’s internal and 
external constituents on the effectiveness of all campus 
programs and services. Highlighting the importance of 
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student satisfaction to student success, Walker-Marshall 
and Hudson (1999) reported that satisfied students were 
more likely to be successful and students with high levels 
of satisfaction in their freshman year were more likely to 
persist in college. 

Using self-reported data in assessment . Using 
senior survey data as a vehicle for assessment raises an 
important methodological issue, i.e., the assumption that 
self-reported data is a valid measure of college impact. 
For some time, researchers have debated the validity of 
this assumption. 

Several early studies yielded evidence of a positive, 
significant relationship between self-reported data and 
tested knowledge. For example, Berdie (1971) found 
moderate to strong correlations between what students 
said they knew and what they actually knew as measured 
by an achievement test. His findings supported the 
conclusion that asking people whether they possess 
information may be a satisfactory method for testing their 
knowledge, particularly with respect to knowledge of 
authors and public figures. Similarly, Baird (1976) 
discovered substantial and significant correlations 
between self-ratings of writing, reading, mathematical 
and scientific ability and scores on various graduate and 
professional school admission tests. In his discussion on 
student outcomes research, Pace (1985) claimed that 
college students are generally accurate reporters and 
their judgments of what they have gained are consistent 
both with external evidence, when it exists, and with what 
we might expect based on their activities and interest. 

Also offering evidence to support the validity of self- 
reported data, Anaya (1999) found that comparable results 
were obtained when using student reported cognitive 
growth with college GPA and standardized test scores, 
and therefore concluded that self-reported gains are valid 
measures that can be used as proxies for more direct 
measures of learning. Hill, Perry and Stein (1998) assert 
that self-reported data, in combination with other 
assessment techniques, can provide useful and important 
information at a relatively low cost on a broad spectrum 
of topics. They also maintain that surveys which focus 
on students’ self-perception of their educational 
experiences can provide insightful measurements and 
help to gauge whether skills taught have been internalized 
by the students. Expressing a similar view, Glynn and 
Rajendran (1993) claim that student perceptions provide 
for a r icher, m ore in elusive o utcomes assessment 
process. They present their perceptual-based study of a 
marketing curriculum to illustrate how incorporating 
students’ perceptions of the relevance of the curricular 
material into the outcomes assessment process can 
provide valuable insight into curricular strengths and 
weaknesses. 



However, Pike (1995) offers some qualification 
regarding the relationship between self reports and 
traditional achievement measures. Based on his multitrait- 
multimethod analyses of self reports and test scores, he 
concludes that using self reports as general indicators of 
achievement can be justified, but substituting self reports 
for test scores cannot be justified. Pike (1999) cites another 
limitation of self-reported data. He asserts that “halo 
error” may be an important component in students’ ratings 
of their learning and development and that “halo error” 
may obscure relationships between college experiences 
and educational outcomes. Pascarella (2001) contends 
that self-reported gains in growth do not permit the same 
level of internal validity as does assessing gains with a 
pretest-posttest design. He proposes introducing a control 
variable, a measure of students’ openness or receptivity 
to e ducational e xperiences w hen t hey e nter c ollege. 
Pascarella suggests that self-reported data may then 
represent a convenient alternative to more expensive 
and time-consuming standardized tests as a method for 
estimating the impact of college on students. 

Understanding differences in assessment bv 
student characteristics. Previous research suggests 
that gender and citizenship may account for significant 
differences in students’ college experience. With respect 
to gender, Hearn (1985) found that faculty availability and 
stimulating course work were more critical for women 
than men as determinants of graduating seniors’ overall 
evaluation of the academic program. Tatra (1995) identified 
gender effects on student evaluation of faculty, with 
females giving higher ratings than males. Lee, Keough 
and Sexton (2002) discovered that, compared with 
women, men were more likely to negatively appraise the 
campus climate and social connectedness was more 
related to lower stress levels among men. 

With regard to citizenship, Delaney (2002) reported 
that international students attributed more importance to 
artistic and intellectual goals. They were more interested 
in achieving in a performing art and in creating artistic 
work and put more emphasis on a general education as 
a reason for attending college. In terms of activities and 
interests, international students spent more time talking 
with teachers outside of class and studying or doing 
homework, while domestic students spent more time 
socializing with friends and engaging in exercise and 
sports. Other research highlights the importance of 
language proficiency to international students’ adjustment 
to U.S. campuses. Bunz (1997) describes the change in 
language as the most important adjustment for 
international students from Western European countries 
attending American universities. Echoing similarfindings, 
Zimmerman (1995) and Nicholson (2001) identified a 
deficiency in language proficiency as an impediment to 
international students’ adjustment to American culture. 
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Identifying predictors of overall satisfaction and 
re-evaluation of college choice . Understanding what 
factors influence overall satisfaction is crucial to promoting 
continuous improvement both at the institutional and 
student level. Examining a variety of factors, Browne, 
Kaldenberg, Browne, and Brown (1998) found that global 
satisfaction with college tended to be driven by the 
student’s assessment of the quality of the course work 
and other curriculum related factors, while another set of 
factors, involving interaction with personnel and how 
students were treated, was more predictive of whether or 
not a student would recommend a college to a friend. 
Franklin (1994) identified satisfaction with one's major; 
perceived g rowth i n p ersonal d evelopment; e nhanced 
analytical p roblem s olving s kills; a nd s atisfaction w ith 
individual components of the educational experience as 
predictors of overall satisfaction. Franklin and Knight 
(1995) discovered that students use the following criteria 
to determine their satisfaction with higher education: 
finding pride and inner satisfaction with accomplishments; 
a flexible curriculum; a university with status and prestige; 
a university degree that leads to career opportunities; the 
encouragement of student involvement; a caringfaculty; 
an opportunity for independence; a student-oriented 
university administration; and a university experience that 
provides an opportunity for growth and development. 
Elliott and Healy (2001) found that performance gaps - 
differences between importance and satisfaction - on 
student centeredness, campus climate and instructional 
effectiveness were strong and significant predictors of 
students’ overall satisfaction. 

Lamport’s (1993) review of the literature highlights the 
importance of student-faculty interaction to students' 
overall satisfaction with college. This body of research 
indicates that interaction provides opportunities for 
personal encounters and for sharing work and ideas that 
yield satisfying experiences for students and faculty. More 
recently, Kuh and Hu (2001) confirmed that both 
substantive and out-of-class contacts with faculty positively 
influenced what students get from their college experience, 
their views of the college environment, and their 
satisfaction. Their research, with over 5,000 students 
from 126 colleges and universities, also revealed that the 
positive effects of student-faculty contact on satisfaction 
and gain were mediated through the effects that students 
expend in other activities. Students who were better 
prepared academically tended to interact more frequently 
with faculty. 

The present study incorporates many of the variables 
identified in the literature in order to better understand 
what factors impact students’ growth and what 
experiences affect their overall satisfaction. The study 
represents an effort to utilize senior survey research as 
part of an assessment program. 



Methodology 

Data Source . Findings in this study are based on 
analyses of senior survey results for the graduating 
classes at a private, selective business college in the 
Northeast. The college enrolls approximately 1,700 
undergraduate students from most of the 50 states, many 
U.S. territories, and some 70 other countries. 
Approximately 39 percent of the undergraduate population 
are female and about 20 percent are international 
students. 

The data include responses from 1997, 1998, 1999 and 
2001 graduating seniors, because a similar survey was 
used during these years. The total number of cases is 
970, representing a response rate of 67 percent. Annual 
response rates ranged from 64 percent in 1999 to 71 
percent in 2001 . Data were collected by mail from 1997 
through 1999 and by mail and on-line in 2001. 

Research Model . The study involves the application 
of a statistical and evaluation model designed to expand 
students’ voice in assessment and achieve optimum 
impact on policy through effective use of senior survey 
research. This model involves the following steps: 

• Review the institutional mission. 

• Identify the goals of the undergraduate academic 
program. 

• Define the major components of the undergraduate 
student life experience. 

• Develop a means to evaluate academic goal 
achievement and satisfaction with student life. 

• Design a statistical analysis plan to address planning 
and policy issues. 

• Translate the results into recommendations for 
planning and policy development. 

The following discussion describes how this model 
was applied in the present study. 

1. Review the institutional mission. A college mission 
statement reflects the institution’s vision and values. It 
serves as a focal point for curricular and program planning 
and therefore provides a useful reference in an 
assessment study. The study College’s mission is to be 
an internationally recognized leader in management 
education. Through its programs and practices, the 
College educates innovative leaders capable of 
anticipating, initiating, and managing change. In a climate 
of entrepreneurial spirit, creative and analytical thinking, 
global p erspectives, c ontinuous I earning a nd s ocial 
responsibility, m en a nd w omen o f d ifferent c ultures, 
origins, and life stages learn together to define the 
opportunities of the future. The goals articulated in this 
mission statement are reflected in the undergraduate 
curriculum, and several were operationally defined in the 
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senior survey. Graduating seniors were asked to evaluate 
to what extent their education enhanced their ability to 
lead, be an effective team member, develop awareness 
of social problems, formulate creative ideas, acquire new 
skills and knowledge, and think analytically. 

2. Identify the goals of the undergraduate academic 
program. Undergraduate program publications were used 
as the source for identifying the major goals of the 
academic program. The following five major competencies 
of the undergraduate curriculum defined the goals: 
leadership/teamwork/creativity, rhetoric, numeracy and 
technology, ethics and social responsibility, and 
international/multi-cultural perspectives. Specific 
questionnaire items were generated to represent these 
competencies. Additional items were created to address 
more general, intellectual and personal growth goals of a 
college education. 

3. Define the major components of the undergraduate 
student life experience. College publications were used 
to identify the major components of undergraduate student 
life, including the academic program, residential life, 
campus life, personal development opportunities, student 
services and other college resources. 

4. Develop a means to evaluate academic goal 
achievement and satisfaction with student life . A 
customized senior survey was designed for this study. 
The major section focused on students’ evaluation of 
their education with respect to goals accomplished and 
level of satisfaction experienced. Specific questionnaire 
items were created in each of these major areas. For 
example, academic experience items included the quality 
of course instruction, faculty attitude and availability, 
academic advising, and availability of courses. Additional 
questions solicited information about students’ 
participation in college experiences, their career values, 
educational and career plans, and selected demographic 
and academic background information. The majority of 
items were common to a consortium senior survey, thus 
providing a basis for comparison with peer institutions. 

5. Design a statistical analysis plan to address planning 
and policy issues. The analyses followed a systematic 
plan, focusing first on univariate results in summary form. 
Next, bivariate analyses were conducted to identify 
relationships between individual student characteristics 
and assessment and satisfaction. Analysis of variance 
was employed to answer the following questions: How 
have graduating seniors’ evaluation and levels of 
satisfaction changed over time? Are there significant 
differences in evaluation and satisfaction between male 
and female and between domestic and international 
students? Multiple regression was used to predict overall 
satisfaction and to simulate how improvements in one 
area might increase overall satisfaction. Logistic 
regression was utilized to predict the probability of 
choosing the same institution and to assess how changes 



in one area would affect students’ decision to choose the 
same college again. 

6. Translate the results of the study into 
recommendations for planning and policy development. 
To ensure an impact on policy, results from this study 
were translated into 12 strategic policy recommendations. 
Five recommendations proposed initiatives that might be 
developed to address areas of weakness and seven 
recommendations suggested ways positive results might 
be used to exploit the institution’s strengths. 

Results 

Significant Differences in Assessment bv Year. 

Data presented in Table 1 reflect graduating seniors’ 
assessment of the College’s influence on their 
competencies and intellectual and personal growth. 
Several of the means show slight to moderate decreases. 
In addition, the differences between some means are 
statistically significant. Three of the decreases may be 
the result of changes in wording in the 2001 survey: from 
team membership to cooperativeness; from use 
quantitative tools to math ability; and from oral 
communication to public speaking. Other statistically 
significant differences may be because of the large 
number of cases. Therefore, the analysis focuses 
primarily on overall results. 

The overall data, summarized in the total column, 
reveal that graduating seniors think their college education 
had a very positive influence on their leadership/ 
teamwork/creativity competency; the three total mean 
ratings exceed 3.0. Next in order, graduating seniors 
consistently report the highest ratings on the rhetoric 
competency, particularly oral communication, while they 
report lower ratings on both the ethics and social 
responsibility and international/multi-cultural perspective 
competencies. Within the numeracy and technology 
competency, students consistently rate their education 
very highly on the ability to use computers, while they 
report lower ratings for the effect of their education on 
their quantitative skills and ability to understand the 
process of science. 

Results presented in Table 1 also show that seniors 
consistently evaluate their college education very positively 
regarding its effect on their intellectual growth. From 1997 
through 2001 , they report mean ratings close to 3.30 or 
higher for the impact of their education on their ability to 
acquire new skills and knowledge; gain in-depth 
knowledge; and think analytically and logically. The 
increased ratings for acquiring new skills and knowledge 
and decreased ratings for gaining in-depth knowledge of 
a field are consistent with a change in emphasis from the 
old to the new curriculum. The new curriculum replaced 
majors w ith a n i ntegrated a pproach t o I earning a nd 
intensified focus on developing students’ capacity for 
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continuous learning. The Class of 2001 was the first 
class to experience the new curriculum. With regard to 
personal growth, students perceive a moderately positive 
effect on the ability to understand themselves, but very 
little effect on their ability to appreciate the arts. 

Table 1 

Annual Assessment of the College's Influence on 
Student Competencies 



Competencies 


1997 


1998 


1999 


2001 


Total 


F Ratio 


A. Curriculum Competencies 
Leadership/Teamwork/Creativity 

Lead and Supervise 3.35 3.47 3.35 


3.33 


3.37 


1.64 


Team Member/Cooperativeness+ 


3.56 


3.48 


3.40 


3.11 


3.37 


19.21*** 


Formulate Creative Ideas 


3.19 


3.22 


3.05 


2.95 


3.10 


6.24*** 


Numeracy and Technology 

Use Computers 


3.53 


3.55 


3.48 


3.52 


3.52 


.36 


Use Quantative Tools/Math Ability+ 


2.90 


3.18 


2.99 


2.53 


2.89 


27.41*** 


Understanding the Process of Science 


- 


1.75 


1.68 


2.06 


1.84 


17.69*** 


Rhetoric 

Oral Communication/Public Speaking+ 


3.44 


3.44 


3.30 


3.24 


3.35 


5.13** 


Write Effectively 


3.10 


3.07 


2.91 


2.98 


3.01 


2.69* 


Ethics and Social Responsibility 

Identify moral and ethical issues 


2.71 


2.60 


2.48 


2.63 


2.61 


2.75* 


Develop awareness of social problems 


- 


2.38 


2.26 


2.34 


2.33 


1.04 


International/Multi-Cultural Perspective 

Relate well to people of diff. races 


2.96 


3.02 


2.76 




2.92 


5.02** 


Read or speak a foreign language 


1.49 


1.56 


1.46 


1.65 


1.55 


1.92 


B. Intellectual and Personal Growth 

Intellectual Growth 

Acquire new skills and knowledge 3.37 3.32 3.28 


3.62 


3.41 


13.78*** 


Gain in-depth knowledge of a field 


3.41 


3.39 


3.30 


3.26 


3.34 


2.43 


Think analytically and logically 


3.46 


3.43 


3.29 


3.25 


3.35 


5.25*** 


Personal Growth 

Understand myself 


3.19 


3.16 


3.07 


2.94 


3.08 


4.70** 


Appreciate the arts 


1.98 


1.90 


1.82 


2.12 


1.96 


5.38*** 



Notes: These mean ratings are based on the following scale: 1 'not at all', 2 'a little', 3 
'moderately 1 , and 4 'greatly 1 . 

+ The second item was used in the 2001 survey. 

* p < .05 ** p < .01 *** < .001 



Figure 1 graphically summarizes graduating seniors’ 
assessment of the impact of undergraduate education 
on students’ competencies and growth. Based on relative 
gaps in the total mean ratings for the four years, 
competencies are classified in three categories indicating 
the perception of a strong, moderate, or weak effect. As 
shown, the perceived effect is strong for the effect of 
undergraduate education on the ability to use computers, 
and on enhanced ability to acquire new skills and 
knowledge, think analytically, lead and supervise, be an 
effective team member, speak in public, and gain in- 
depth knowledge of a field. In marked contrast, the 



perceived effect is weak for the effect of undergraduate 
education on understandng of ethics, awaeness of social 
problems, ability to appreciate the arts, and read or 
speak a foreign language. 

Several of the competencies rated highly by students 
- including the ability to lead and supervise, be an effective 
team member, acquire new skills and knowledge, and 
think analytically and logically - relate to the College’s 
mission to prepare innovative leaders with a commitment 
to continuous learning. 

Figure 1 

Overall Assessment of College Impact on 
Student Competencies 

0 0 0.5 1.0 1.5 2.0 2 5 3.0 3.5 4.0 

Use Computers 3.52 

Acquire new skills knowledge 3 

Supervise 3 37 

Team Member/Cooperativoness 3.37 

analytically and logically 3 35 

Oral Communication/Public Speaking 3 35 

Gam in -depth knowledge field 3.34 

Strong 



Formulate Idea-? 3.10 

Understand myself 1 08 

Write Effectively 3 

Relate to people of diff race! 2 

Use Quantative Tools/Math Ability 2.89 

Moderate 



Identify moral ethical issue* • 

Develop awareness of social problems 2 33 

Appreciate the arts ■■■■■■■■■ 1 96 
Understanding the Process of Science i 84 

Read or speak a foreign language ^ I 55 Weak 

Significant Differences in Assessment bv Gender 
and Citizenship . Two-way analyses of variance were 
conducted to examine the relationship between gender 
and citizenship and graduating seniors’ assessment of 
the impact of their education. Results from these analyses, 
identifying main and interaction effects, are presented in 
Table 2. 

As shown, female graduating seniors reported 
significantly higher ratings for the effect of their education 
on four competencies and areas of growth. The largest 
difference occurs with respect to ‘understanding myself.’ 
Female seniors report a mean of 3.21, compared with 
2.98 for the males. While both males and females report 
high ratings for the effect of their education on leadership 
and oral communication, female ratings are significantly 
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Table 2 

Significant Differences by Gender and Citizenship in 
Assessment of College Impact 



Mean Ratings 


Gender Differences 


Female 


Male 


Difference 


F ratio 


Lead and Supervise 


3.46 


3.30 


.16 


3.86* 


Oral Communication/Public Speaking + 


3.42 


3.28 


.14 


3.95* 


Read or Speak a Foreign Language 


1.64 


1.46 


.18 


4.49* 


Understand Myself 


3.21 


2.98 


.23 


5.60* 


Mean Ratings 


Differences by Citizenship 


International 


U.S. Citizen 


Difference 


F ratio 


Read or Speak a Foreign Language 


2.17 


1.42 


.75 


86.72*** 


Write Effectively 


3.17 


2.99 


.18 


5.93* 


Appreciate the Arts 


2.17 


1.93 


.24 


9.10** 


Acquire New Skills and Knowledge 


3.53 


3.38 


.15 


5.94* 


Mean Ratings 


Interaction by Gender * Citizenship 


Female 


Male 


F ratio 


Formulate Creative Ideas 








7.97** 


International 


2.98 


3.28 






U.S.Citizen 


3.14 


3.03 






Gain In-depth Knowledge 








4.45* 


International 


3.22 


3.45 






U.S.Citizen 


3.36 


3.31 






Notes: These mean ratings are based on the following scale 


1 'not at all', 2 


'a little', 3 'moderately 1 , 


and 4 'greatly 1 










+ The second item was used in the 2001 survey. 








* p < .05 ** p < .01 *** p < .001 











higher than those of males on these competencies. In 
comparison, although both mean ratings are relatively 
low, females report a higher mean for the effect of their 
education on their ability to learn to read and speak a 
foreign language. 

As documented in Table 2, the perceived effect of 
undergraduate education on certain competencies also 
varies by citizenship. As shown, international students 
reported significantly higher ratings than domestic 
students for the effect of their college education on four 
abilities. The largest difference involves a higher rating 
for enhancing the ability to read or speak a foreign 
language. International students also report significantly 
higher ratings for the impact of college on their abilities 
to write effectively, appreciate the arts, and acquire new 
skills and knowledge. 

Analyses of variance identified interaction effects 
between gender and citizenship on two variables: the 
ability to formulate creative ideas and to gain in-depth 
knowledge. In both instances, female international 
students reported the lowest mean and male international 
students reported the highest mean rating. 



Significant Differences in Satisfaction bv 
Year \ The senior survey explored not only 
how much students thought they learned but 
also how satisfied they were with numerous 
facets of the academic program and student 
life during the four-year period. Analyses of 
variance identified several statistically significant 
differences by year. In some instances the 
data show a fluctuating pattern, while in other 
cases the data indicate a definite trend. Results 
are presented in Table 3 (page 8). 

As shown, in the academic area, satisfaction 
ratings are consistently high for the quality of 
business courses and for faculty availability 
and faculty attitude. In contrast, satisfaction 
ratings are lower for humanities, science and 
math and social science courses; this difference 
may be attributed to student lack of interest in 
these areas. The notable increase in 
satisfaction with academic advising may be 
related to the implementation of a new advising 
program during this time. Through the Learning 
Plan Program, students met with a faculty 
member and an outside mentor a number of 
times during the semester in their junior year to 
develop an academic, co-curricular and career 
plan. 

Significant differences by year were found in 
four areas of student life: residential life, campus 
life, personal development opportunities, and 
resources and student services. With respect 
to residential life, trend data reveal that 
satisfaction improved for campus safety, while 
satisfaction declined regarding housing and 
food services. The decline in student satisfaction with 
housing may be related to higher than expected yields in 
the number of enrolled students and the consequent 
overcrowding and use of converted space for housing. 
The decrease in satisfaction with food services may 
indicate dissatisfaction with the meal plan rather than the 
food. Issues raised by the students include the price of 
the meal plan and restrictions on using certain cafeterias. 
Subsequently, adjustments were made in response to 
student concerns. 

The most remarkable trend in the area of campus life 
was the significant increase from 1997 to 1998 in high 
satisfaction ratings for the campus student center, 
following the construction of a new building. Further, the 
mean ratings remained consistently high after the opening 
of the new center. In contrast, satisfaction ratings 
decreased significantly from 2.77 to 2.54 for the inclusion 
of student voice in policies. Two factors may account for 
this trend. During this time period, the College gave less 
attention to including students on college committees. 
Second, student government leaders expended most of 
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Table 3 

Seniors' Satisfaction with Academic Experiences and Student Life 





1997 


1998 


1999 


2001 


Total 


F Ratio 


Academic Experiences 














Business Courses 


3.69 


3.66 


3.64 


3.73 


3.68 


1.19 


Science and Math Courses 


2.97 


3.01 


2.85 


2.87 


2.92 


3.18* * 


Humanities and Arts Courses 


2.87 


2.99 


2.96 


2.87 


2.92 


1.80 


Social Sciences Courses 


2.88 


2.94 


2.81 


2.77 


2.85 


3.22* 


Faculty Availability 


3.48 


3.51 


3.51 


3.54 


3.51 


.48 


Faculty Attitude 


3.42 


3.47 


3.43 


3.38 


3.42 


.77 


Independent Study 


3.26 


3.19 


3.16 


3.35 


3.23 


1.96 


Academic Advising 


2.80 


3.02 


2.97 


3.05 


2.97 


5.16** 


Availability of Courses 


2.92 


2.84 


2.91 


2.75 


2.85 


2.84* 


Residential Life 














Campus Safety 


- 


3.00 


3.01 


3.37 


3.14 


23.63*** 


Housing 


- 


3.14 


3.13 


2.96 


3.07 


5.54** 


Food Services 


3.10 


3.01 


3.13 


2.90 


3.02 


5.19*** 


Campus Life 














Campus Student Center 


2.35 


3.30 


3.21 


3.26 


3.09 


85.21*** 


Climate for Minorities 


2.78 


2.90 


2.81 


3.02 


2.88 


4.41** 


Ethnic/Racial Diversity 


- 


3.05 


2.91 


3.04 


3.00 


2.41 


Social Life on Campus 


2.49 


2.51 


2.44 


2.43 


2.47 


1.08 


Student Government 


2.75 


2.73 


2.75 


2.68 


2.73 


.50 


Student Voice in Policies 


2.77 


2.74 


2.56 


2.54 


2.65 


7.22*** 


Personal Development Opportunities 














Recreation/Athletics 


3.19 


3.18 


3.12 


3.06 


3.13 


1.74 


Extra-Curricular Programs 


- 


3.22 


3.31 


2.93 


3.13 


21.45*** 


Religious/Spiritual Life 


- 


2.88 


3.04 


2.71 


2.86 


10.51*** 


Career Planning 


2.96 


2.95 


3.02 


2.84 


2.94 


2.60 


Resources and Student Services 














Classroom Facilities 


3.56 


3.61 


3.58 


3.52 


3.56 


1.02 


Library Resources 


3.10 


3.17 


3.16 


3.27 


3.18 


2.79* 


Computer Services 


3.18 


3.32 


3.45 


3.11 


3.25 


11.32*** 


Financial Aid Services 


2.67 


2.79 


2.89 


2.96 


2.84 


5.89*** 


Health Services 


3.04 


2.87 


3.06 


3.16 


3.04 


7.16*** 


Counseling Services 


2.86 


3.00 


2.89 


2.96 


2.93 


1.12 



Note: These mean ratings are based on the following scale: 2 'very or generally dissatisfied', 3 'generally 
satisfied', and 4 'very satisfied'. 

* p < .05 ** p < .01 



p < .001 
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their efforts on social activities, such as sponsoring parties. 
Therefore, students felt they had little voice in the 
development of college policy. 

Significant differences in satisfaction ratings over time 
were also found with two personal development 
opportunities, extra-curricular programs and religious/ 
spiritual life. A review of the data reveals moderate and 
somewhat fluctuating levels of satisfaction with extra- 
curricular programs and the college’s religious/spiritual life. 

In t he a rea o f r esources a nd s tudent s ervices, 
satisfaction ratings were consistently positive for computer 
services, and mostly positive for health services. Ratings 
increased for financial aid services. This may be the 
result of a change in policy from allowing a gap in meeting 
student need to meeting students’ full need. Several 
factors may have contributed to the consistently positive 
and increasing ratings for library resources, including the 
highly qualified and dedicated library staff and specific 
developments related to customer service. In the mid 
I990’s, as part of a Total Quality Management effort, the 
library staff implemented the following initiatives to better 
serve students: a single service point; a process for 
students to sign up for appointments for reference 
consultations; and an enhanced liaison program, with 
each librarian assigned to a division and a functional 
group. With the establishment of a new integrated 
undergraduate curriculum in 1996, considerable effort 
was expended on developing a systematic library 
instruction program for undergraduates starting in the 
freshman year. Also during this time period more library 
resources became available electronically and campus- 
wide, and the collection budget was increased $100,000 
in 1997 to recognize the need for electronic resources. 

Figure 2 d isplays g raduating s eniors’ o verall m ean 
satisfaction ratings with various aspects of the academic 
program, including courses, faculty attitude, and the 
availability of courses. These ratings represent the 
average for the four years of the study. As shown, these 
data document the highest level of satisfaction with the 

Figure 2 

Graduating Seniors’ Satisfaction with 
Academic Experiences 

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 



Quality of Business Instruction 3.68 

Faculty Availability 3.51 

Faculty Attitude 3.42 

Independent Study 3.23 

Academic Advising 2.97 

Quality of Science & Math Instruction 2.92 

Quality of Humanities and Arts Instruction 2.92 

Quality of Social Sciences Instruction 2.85 

Availability of Courses 2.85 



quality of business course instruction (3.68), followed by 
faculty availability (3.51), faculty attitude (3.42), and 
independent study (3.23). As indicated by the relative 
gap in mean ratings, satisfaction is lower with respect to 
other aspects of the academic program, including 
academic advising (2.97) and availability of courses (2.85). 

Figure 3 displays graduating seniors’ overall satisfaction 
with various aspects of student life. These data reflect a 
high level of satisfaction with campus resources, 
particularly classroom facilities (3.56), computer services 
(3.25), and library resources (3.18). Results also indicate 
moderate s atisfaction w ith m any s tudent s ervices, 
including housing, health and food services. In contrast, 
the data reflect much lower satisfaction with many aspects 
of campus life, including student government (2.73), 
student voice in policies (2.65), and campus social life 
(2.47). 

Figure 3 

Graduating Seniors’ Satisfaction with 
Student Life 

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 



Classroom Facilities ! 
Computer Services 
Library Resources 1 
Campus Safety 
Recreation/Athletics ! 
Extra-Curricular Programs 

Campus Student Center 
Housing 1 


























Career Planning 1 










Religious/Spiritual Life 














Social Life on Campus i 





Note These mean ratings are based or the following scale 2 very or generally dissatisfied: 3 'generally satisfied 
and 4 very satisfied 

Significant Differences in Satisfaction bv Gender 
and Citizenship. Two-way analyses of variance were 
also c onducted t o e xamine t he r elationship b etween 
gender and citizenship and graduating seniors’ satisfaction 
with aspects of their undergraduate experience. Results 
from these analyses are presented in Table 4 (page 10). 

As shown in Table 4, two-way analyses of variance 
identified several statistically significant differences by 
gender and citizenship in seniors’ satisfaction with aspects 
of their undergraduate experience. Four differences were 
found by gender, and all the female ratings on these 
items are higher than the male ratings. The largest 
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Table 4 

Significant Differences by Gender and 
Citizenship in Seniors' Satisfaction with College 



Mean Ratings 


Undergraduate Experience 


Female 


Male 


Difference | 


F ratio 


Quality of Humanities and Arts Courses 


3.09 


2.85 


.24 


4.91* 


Campus Student Center 


3.37 


3.18 


.19 


7.95** 


Financial Aid Services 


2.93 


2.74 


.19 


4.19* 


Social Life on Campus 


2.56 


2.39 


.17 


4.01* 


Undergraduate Experience 


International | 


U.S. Citizen 


Difference | 


F ratio 


Ethnic/Racial Diversity 


3.25 


2.94 


.31 


8.49** 


Climate for Minorities 


3.11 


2.85 


.26 


6.43* 


Academic Advising 


3.20 


2.97 


.23 


5.80* 


Faculty Attitude 


3.57 


3.39 


.18 


4.36* 


Food Services 


2.84 


3.03 


-.19 


6.30* 


Recreation/Athletics 


2.90 


3.10 


-.20 


4.12* 


Student Health Services 


3.20 


2.99 


.21 


3.90* 


Financial Aid Services 


3.04 


2.80 


.24 


4.81* 


Note: These mean ratings are based on the following scale: 
'generally satisfied', and 4 'very satisfied'. 


2 'very or generally dissatisfied', 3 


* p < .05 ** p < .01 *** p < .001 











difference reflects a higher level of satisfaction among 
females for the quality of humanities and arts courses. 
Females are also more satisfied than males with the 
campus student center, financial aid services, and campus 
social life. 

Analyses of variance also revealed that international 
and domestic students differ significantly in their level of 
satisfaction with certain aspects of their undergraduate 
experience. As shown in Table 4, international students 
report significantly higher levels of satisfaction with two 
facets of their academic experience - academic advising 
and faculty attitude and with several aspects of campus 
life and student services, including ethnic/racial diversity, 
the climate for minorities, health and financial aid services. 
In contrast, U.S. citizens report higher levels of satisfaction 
with food services and recreation/athletics programs. 

Predicting Overall Satisfaction . Multiple regression 
was employed to answer the question: What are the 
significant predictors of gr aduating seniors’ overa II 
satisfaction? Based on previous research and bivariate 
analyses in the present study, selected demographic, 
assessment and satisfaction variables were chosen as 
potential independent variables. V ariables identified in 
previous studies include students’ assessment of the 
quality of the coursework (Kaldenberg, Browne, & Brown, 
1998); satisfaction with one’s major, perceived growth in 
personal development and enhanced analytical problem 
solving skills (Franklin, 1994); a degree that leads to 
career opportunities and a university experience that 
provides an opportunity for growth and development 
(Franklin & Knight, 1995); and student-faculty interaction 



(Lamport, 1993; Kuh & Hu, 2001). Several variables 
measuring these same constructs were selected as 
predictors in this study if they met the statistical criterion, 
i.e. , a correlation coefficient > .20, at the .001 level of 
significance. Gender and citizenship were also included 
based on the institution’s policy interest in these variables. 

Prior to conducting the regression, two analyses were 
performed to investigate the extent of multicollinearity 
among the independent variables. Correlational analyses 
revealed relatively small correlations among the 
independent variables. Fifty percent of the coefficients 
were less than .20 and only one coefficient exceeded .40. 
Tolerance statistics were also computed for each of the 
independent variables. The tolerance statistic represents 
the proportion of a variable’s variance not accounted for 
by other independent variables in the equation. 2 The 
tolerance coefficients in this study ranged from .64 to .98, 
indicating that these variables are substantially unique. 

Regression results are displayed in Table 5. As shown, 
the two demographic variables - gender and citizenship 
- do not significantly predict overall satisfaction. In contrast, 
all of the seven assessment and satisfaction variables do 
significantly predict overall satisfaction. As indicated by 
the coefficients, the strongest predictor of students’ overall 
satisfaction is perceived gain in in-depth knowledge of a 
field ( b=.212). Next in order are satisfaction with the 
quality of business courses (b=. 139) and satisfaction 
with faculty attitude (b=.107). In addition to satisfaction 
with career services (b=.065), other significant predictors 
involve general goals of a college education, i.e., to 
enhance students’ ability to acquire new skills and 
knowledge ( b=.097); understand moral/ethical issues 
(b=.093); and think analytically ( b=. 077). The R 2 of .25 
indicates that the model explains 25 percent of the 
variance in students’ overall satisfaction. 

Results from the regression were employed to simulate 

Table 5 

Multiple Regression Results: 

Predicting Graduates' Overall Satisfaction 



Variables 


Beta 

Coefficient 


t 

Ratio 


R 2 


F Ratio 


Demographic Variables 










Gender 


.056 


1.802 






Citizenship 


.003 


0.107 






Assessment and Satisfaction Variables 










Gained in-depth knowledge 


.212 


5.815*** 






Satisfaction with business courses 


.139 


3.965*** 






Satisfaction with faculty attitude 


.107 


3.208*** 






Enhanced ability to acquire new knowledge 


.097 


2.652** 






Enhanced understanding of ethical issues 


.093 


2.764** 






Enhanced ability to think analytically 


.077 


2.006* 






Satisfaction with career services 


.065 


2.002* 












.250 


29.423*** 


* p < .05; ** p < .01 ; *** p < .001 
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how improvements in specific areas would affect overall 
satisfaction. The simulation was conducted to 
demonstrate to administrators what effect improvements 
in certain areas would have on students’ overall 
satisfaction. Because most of the original predictor mean 
ratings in this data set were high, the means for three 
independent variables were increased to 3.50. Mean 
ratings were increased from 3.32 to 3.50 for gained in- 
depth knowledge, from 2.59 to 3.50 for enhanced 
understanding of moral and ethical issues, and from 2.93 
to 3.50 for satisfaction with career services. The value 3.50 
represents the midpoint between 3 ‘moderately’ and 4 
‘greatly’ for the first two variables and 3.50 represents the 
midpoint between 3 ‘generally satisfied’ and 4 Very satisfied’ 
for the third variable, satisfaction with career services. 

Simulation results are presented in Table 6. The table 
presents two sets of means, original and revised, for the 
independent variables and the resulting increase in the 
dependent variable. 

The original means are based on the actual data. 

Each unweighted mean is multiplied by the unstandardized 
B coefficient to produce the weighted mean. For example, 
the 3.32 for gained in-depth knowledge is multiplied by 
the B coefficient of .203 to produce a weighted mean of 
.674. The weighted means, with the intercept, are used 
to compute the regression equation, resulting in a mean 
of 3.1 8 for overall satisfaction. 

The revised column means (shown underlined) 
represent the simulated increases from 3.32 to 3.50 for 
gaining in-depth knowledge, from 2.59 to 3.50 for 
enhanced understanding of moral and ethical issues, 
and from 2.93 to 3.50 for satisfaction with career services. 
The revised unweighted mean of 3.50 for gaining in- 
depth knowledge is multiplied by the unstandardized B 



coefficient of .203 to produce a weighted mean of .711. 
Similarly, the revised unweighted mean of 3.50 for 
enhanced understanding of moral and ethical issues is 
multiplied by the B coefficient of .076 to produce a 
weighted mean of .266. As shown, increasing the means 
for gaining in-depth knowledge, enhanced understanding 
of ethical issues and satisfaction with career services 
increases the overall mean satisfaction rating from 3.18 
to 3.33. 

Predicting Choice of Same Institution. Logistic 
regression was employed to test a model predicting the 
probability of graduating seniors’ choosing the same 
institution again. The model includes nine variables 
representing demographic characteristics, academic 
achievement, perceived assessment of college impact 
and s atisfaction w ith various a spects of t he c ollege 
experience. These variables were chosen based on the 
literature as well as on results from bivariate analyses in 
the present study. Table 7 presents statistical results 
from this analysis, including the coefficient estimates 
with their standard errors, Wald statistics, p-values and 
odds ratios. In addition, the table provides statistics 
assessing the model fit. 

As shown in Table 7, eight of the nine variables included 
in the model were significant predictors of choosing the 
same college. The parameter estimates provide a 
measure of the effect of a one unit change in the predictor 
variable on the outcome or dependent variable. For 
example, a one unit change in students’ rating for 
enhanced ability to develop creative ideas increases the 
log odds of choosing the same institution by .57, or in 
terms of the odds ratio by a factor of 1 .77. 

These logistic regression results reveal that the two 
strongest predictors of students’ 
re-evaluation of their college 
choice are satisfaction with 
campus social life and satisfaction 
with the quality of business 
courses. The odds ratio of 3.14 
for satisfaction with campus life 
indicates that a person who is very 
satisfied with campus life has odds 
of choosing the same institution 
that are 3.14 times higher than a 
person who is satisfied. Similarly, 
a person who is very satisfied with 
the quality of business courses 
has odds of choosing the same 
institution that are 2.48 times 
higher than a person who is 
satisfied. Other significant 
parameter estimates include a 
positive assessment of enhanced 
ability for creative ideas (B=.57), 



Table 6 

Simulation Results: Effect of Selected Increases in Assessment and 
Satisfaction on Overall Satisfaction 



Means 


Variables 


Unstandardized 
B Coefficient 


Origina 

Unweighted 


Weighted 


Revisec 

Unweighted 


Weighted 


Constant 


.298 










Gender 


.079 


1.45 


.115 


1.45 


.115 


U.S. Citizenship Status 


.006 


1.84 


.011 


1.84 


.011 


Gained in-depth knowledge 


.203 


3.32 


.674 


3.50 


.711 


Satisfaction with business courses 


.184 


3.67 


.676 


3.67 


.676 


Satisfaction with faculty attitude 


.126 


3.42 


.431 


3.42 


.431 


Enhanced ability to acquire new knowledge 


.099 


3.39 


.336 


3.39 


.336 


Enhanced understanding of ethical issues 


.076 


2.59 


.197 


3.50 


.266 


Enhanced ability to think analytically 


.078 


3.34 


.261 


3.34 


.261 


Satisfaction with career services 


.063 


2.93 


.185 


3.50 


.221 


Dependent Variable 












Overall Satisfaction 






3.18 




3.33 
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having gained in-depth knowledge (B=.48), satisfaction 
with the availability of courses (B=.37), satisfaction with 
campus safety (B=.48), and high average grades in college 
(B=.28). 

Table 7 

Logistic Regression Results: Predicting 
Whether or Not Students Would Choose the 
Same Institution 





Parameter 


Standard 


Wald 


Odds 


Variables 


Estimate 


Error 


Chi-Square 


Ratio 


Intercept 


-12.760 








Gender 


0.465 


.204 


5.173* 


1.592 


U.S. Citizenship Status 


0.447 


.270 


2.737 


1.564 


Average grade in college 


0.282 


.092 


9.463** 


1.325 


Enhanced ability for creative ideas 


0.569 


.132 


18.418*** 


1.766 


Gain in-depth knowledge 


0.475 


.146 


10.548*** 


1.609 


Satisfaction with campus social life 


1.146 


.199 


33.108*** 


3.145 


Satisfaction with business courses 


0.908 


.206 


19.416*** 


2.479 


Satisfaction with availability of courses 


0.365 


.147 


6.145* 


1.441 


Satisfaction with campus safety 


0.478 


.143 


11.141*** 


1.612 


* p < .05; ** p < .01 ; *** p < .001 










Chi-square: 


211.571, 9 df*** 


Correct Classification Rate: 


74.4 


Nagelkerke R-Square: 


.393 


Sensitivity: 


73.8 


Cutoff (prior probability): 


.60 


Specificity: 


74.7 


Hosmer-Lemeshow goodness-of-fit statistic: 7.618 with 8 df (p < .472) 



The Delta-P statistic can also be computed to determine 
the incremental effect of a unit change in the independent 
variable on the dependent variable. The formula for Delta- 
P is [Odds Ratio/(1 + Odds Ratio)] - P o , where P o = the 
original probability. Using results from this logistic 
regression, one can calculate the effect of a unit change 
in satisfaction with social life on the likelihood of choosing 
the same institution again as follows: Delta-P = [3.145/ 
(1+3.145)] - .72 = .7587-72 = .0387. In other words, for 
every unit change in satisfaction with social 
life, there is a 3.87 percent change in the 
likelihood of choosing the same institution 
again. For further discussion on logistic 
regression, see the publication by Pampel 
(2000), cited in the reference list. 

This model accurately predicted 74 
percent of the cases, 74 percent among 
those who would choose the same 
institution again and 75 percent among 
those who would not choose the same 
institution. With regard to the overall model, 
the Nagelkerke R 2 is .39. The Nagelkerke 
R 2 is a pseudo R 2 , comparable to R 2 in 
multiple regression. The non-significant 
Hosmer and Lemeshow goodness-of-fit 
statistic indicates that the expected and 
observed values are close, suggesting that 
the model is a good fit. 

Coefficients from logistic regression 



were used to simulate the effect of an increase in student 
satisfaction with campus social life on the likelihood of 
choosing the same institution again. This simulation was 
also conducted to demonstrate what impact a change in 
one area might have on the likelihood of students’ 
choosing the same institution again. The mean for 
satisfaction with campus social life was increased from 
2.47 to 3.50, which is considered a high level of 
satisfaction. 

Results are presented in Table 8. The table presents 
two sets of means, original and revised, for the 
independent variables and the resulting probability in the 
dependent variable. The original means are based on the 
actual data. Each unweighted mean is multiplied by the 
unstandardized B coefficient to produce the weighted 
mean. For example, the 2.47 mean for satisfaction with 
campus social life is multiplied by the B coefficient of 
1.146 to produce a weighted mean of 2.828. The weighted 
means, with the intercept, are used to compute the 
regression equation, resulting in the probability of .85 in 
the dependent variable. This finding indicates that seniors 
with average scores on these independent variables have 
a probability of .85 of choosing the same institution again. 

The revised column mean (shown underlined) 
represents the simulated increase in satisfaction with social 
life from 2.47 to 3.50. The revised unweighted mean of 
3.50 is multiplied by the unstandardized B coefficient of 
1.1 46 to produce a weighted mean of 4. 01 1. As shown, the 
effect of increasing the student satisfaction rating with 
social life from 2.47 to 3.50 increases the probability of 
choosing the same college again from .85 to .95. This 
result means that seniors with average scores on these 
variables, including a 3.50 for satisfaction for social life, 
have a probability of .95 of choosing the same institution 
again. 



Table 8 

Simulation Results: Effect of Improvement in Satisfaction with 
Campus Social Life on Decision to Choose the Same Institution 



Means 


Variables 


Unstandardized 
B Coefficient 


Original 

Unweighted Weighted 


Revised 

Unweighted Weighted 


Constant 


-12.76 










Gender 


0.465 


1.45 


0.674 


1.45 


0.674 


U.S. Citizenship Status 


0.447 


1.84 


0.822 


1.84 


0.822 


Average grade in college 


0.282 


3.37 


0.950 


3.37 


0.950 


Enhanced ability for creative ideas 


0.569 


3.10 


1.764 


3.10 


1.764 


Gain in-depth knowledge 


0.475 


3.34 


1.587 


3.34 


1.587 


Satisfaction with campus social life 


1.146 


2.47 


2.828 


3.50 


4.011 


Satisfaction with business courses 


0.908 


3.68 


3.343 


3.68 


3.343 


Satisfaction with availability of courses 


0.365 


2.85 


1.040 


2.85 


1.040 


Satisfaction with campus safety 


0.478 


3.14 


1.503 


3.14 


1.503 


Dependent Variable 












Probability of choosing same college 






.85 




.95 
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Strategic Policy Recommendations 
for the Primary Institution 

Based on the research findings, the study concluded 
with a set of strategic policy recommendations designed 
to further enhance the College’s strengths and identify 
areas for improvement, particularly in the academic 
program and student life areas. Illustrative 
recommendations follow. 

Strengths 

1. Publicize students’ exceptionally high level of 
satisfaction with the faculty and the quality of their business 
courses. 

2. Promote the College’s extraordinary success in 
preparing students to compete in a technologically 
sophisticated business environment. 

3. Continue to offer high quality computer services 
and i nvest i n s tate o f th e a rt I ibrary r esources an d 
classroom facilities. 

4. Sustain and promote the College’s success in 
developing students’ leadership, public speaking and 
teamwork abilities. 

5. S ustain a nd p ublicize t he C ollege’s s trength i n 
preparing students to meet the complex and changing 
demands in their future careers. 

6. Continue the efforts to enhance financial aid services. 

7. In recruiting efforts designed to attract women, 
publicize the undergraduate program’s success in 
developing women’s leadership, teamwork and public 
speaking abilities. 

Areas for Improvement 

8. Investigate the reasons and craft strategies to change 
students’ low level of satisfaction with the social life on 
campus. 

9. Investigate the reasons for decreasing satisfaction 
with student voice in policies. 

10. Evaluate and enhance the effectiveness of the 
curriculum in developing students’ human potential - 
particularly in terms of appreciating the arts, relating with 
diverse groups of people, and having a sense of ethics 
and social responsibility. 

11. Strengthen the emphasis on writing in the 
curriculum. 

12. Review and possibly renew the undergraduate 
curriculum’s emphasis on enhancing students’ ability to 
think creatively. 

Discussion 

The primary focus of this study was on perceived 
effectiveness and satisfaction. However, the data can 
also be used in combination with results from other studies 
to investigate possible relationships between perceived 
effectiveness and importance. Research on the goals of 
entering freshmen at the study institution indicate a 



relationship between graduating seniors’ assessment of 
their education and their priority goals. Cooperative 
Institutional Research Program (CIRP) survey results 
have consistently shown that entering freshmen place a 
high priority on being successful in business, becoming 
an authority in their field, and becoming a community 
leader (Delaney, 1 997 - 2001 ). In this study, graduating 
seniors rated their education highly for developing abilities 
particularly pertinent to these goals: leadership, team 
membership, formulating creative ideas, using computers, 
and public speaking. 

Compared with their peers, entering freshmen at the 
study institution place a very high priority on preparing to 
advance their career and obtain a good job in business, 
and they rate themselves very highly on drive to achieve 
(Delaney, 1997 - 2001). Graduating seniors’ very positive 
evaluation of the college’s impact on their ability to acquire 
new skills, gain in-depth knowledge and think analytically 
and logically suggests that the College is enabling students 
to accomplish their goals for achievement and career 
advancement. 

Results from the freshman and senior studies also 
reflect a similarity between low priority goals and the 
lower evaluation of certain competencies. Entering 
freshmen have consistently expressed little interest in 
personal and social goals, such as developing a 
meaningful philosophy o f life, influ encing social v alues 
and promoting racial understanding (Delaney, 1997 - 
2001). Similarly, graduating seniors report lower ratings 
for the effect of their education on a related set of 
competencies - self-understanding, ethics and social 
responsibility and international/multi-cultural perspectives. 

Results from this study revealed significant differences 
in assessment and satisfaction by student characteristics, 
particularly gender and citizenship. While the primary 
focus and specific findings may vary, previous studies 
have also identified gender differences in students’ 
evaluation of their college experience. For example, Tatro 
(1995) found that females rated faculty higher than did 
males. In this study, females reported higher ratings for 
the effect of their education on their competencies and 
growth; this would suggest a stronger influence by faculty. 
Lee et al. (2002) discovered that males were more critical 
of the campus climate. Similarly, males in this study 
reported a lower rating on campus social life. 

With respect to differences by citizenship, Zimmerman 
(1995), Bunz (1997), and Nicholson (2001) identified 
language as a significant factor in international students’ 
adjustment to American culture, suggesting that learning 
to read and speak a foreign language is critically important 
to i nternational s tudents. D elaney (2 002) f ound that 
international students attributed more importance to artistic 
and intellectual goals and they placed a higher priority on 
gaining a general education as a reason for going to 
college. In this study international students reported 
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significantly higher ratings than domestic students for the 
effect of their education on their ability to read or speak 
a foreign language; write effectively; appreciate the arts; 
and acquire new skills and knowledge. 

Delaney’s (2002) study of entering freshmen revealed 
that international students reported spending significantly 
more time talking with teachers outside of class, studying 
and doing homework, while domestic students spent 
more time socializing with friends and engaging in exercise 
and sports. In this study, international students also 
reported significantly higher satisfaction ratings for 
academic advising and faculty attitude, while domestic 
students reported a significantly higher rating for recreation 
and athletics. Graduating seniors’ significantly different 
ratings may reflect the relative importance international 
and domestic students attribute to academic and social 
experiences. 

Results from this research replicate several previous 
findings regarding factors that influence students’ overall 
satisfaction with the college experience. For example, 
this study identified satisfaction with the quality of business 
courses; satisfaction with faculty attitude; and the 
perception of enhanced ability to think analytically as 
significant predictors of students’ overall satisfaction. 
Similarly, previous studies identified students’ assessment 
of the quality of courses (Browne et al., 1998); enhanced 
analytical problem solving skills (Franklin, 1994); interaction 
with faculty (Lamport, 1993; Kuh and Hu, 2001); and the 
perception of a caring faculty (Franklin & Knight, 1995) as 
determinants of students’ overall satisfaction with the 
college experience. 

This study also identified a different set of predictors 
for the two major dependent variables: overall satisfaction 
and the likelihood of choosing the same college again. 
Two of the significant predictors of the likelihood of 
choosing the same college again were satisfaction with 
campus life and satisfaction with campus safety. These 
variables relate primarily to the quality of one’s personal 
life and interpersonal relationships. Similarly, Browne et 
al. (1998) found that factors involving interaction with 
personnel and how students were treated were more 
predictive of whether a student would recommend a 
college to a friend while assessment of the quality of the 
academic program was more predictive of overall 
satisfaction. 

These findings enhance our understanding of factors 
that can potentially influence students’ overall satisfaction 
and re-evaluation of their college choice. Such information 
may t hen s erve a s a g uide f or d eveloping t argeted 
strategies to impr ove th e college’s performance on 
individual criteria that will impact students’ overall 
satisfaction and regard for the college. 

This study demonstrates the relevance and value of a 
statistical, evaluation model using a senior survey as a 
vehicle for focusing on the students' voice in assessment 



(Lingrell, 1992; Ewell, 1983). This research also illustrates 
how students’ voices can contribute effectively to 
assessment. By expressing different levels of satisfaction 
and perceived growth, students identify potential program 
strengths and areas in need of improvement. While 
researchers differ with regard to the value of self-reported 
data (Pascarella, 2001; Pike, 1999; and Anaya, 1999), the 
author presents this study as evidence that self-reported 
data, in combination with other assessment techniques, 
can provide useful and important information at a relatively 
low cost and can enrich the outcomes assessment 
process (Hill et al., 1998; Glynn & Rajendran, 1993). 

Major components of this model include linking the 
study design to the institution’s mission, the goals and 
structure of the program; developing a structured analysis 
plan to address policy issues; and translating results into 
recommendations for planning and policy development. 
Further, the study illustrates how statistical procedures 
can be used to simulate the effect of a specific 
improvement in one area on overall satisfaction. 
Information derived from such analyses can then guide 
decisions regarding the allocation of efforts and resources 
in those areas that will effect the greatest impact on 
overall student satisfaction, which is crucial to the health 
of the institution and to student success. 
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Endnotes 

’Satisfaction ratings are based on the following scale: 
1 Very dissatisfied’, 2 ‘generally dissatisfied’, 3 ‘generally 
satisfied’, and 4 ‘very satisfied’. Due to the small numbers, 
the ‘very dissatisfied’ and the ‘generally dissatisfied’ were 
combined into one category. 

2 The tolerance statistic is calculated as 1 minus R z for 
an independent variable when it is predicted by the other 
independent variables already included in the analysis. It 
is a measure of the uniqueness of the predictor variables. 
Default tolerance levels range between .01 and .001 
before variables are excluded. For further discussion on 
this topic, see Tabachnick and Fidell (2001, p. 84). 
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